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METHOD AND SYSTEM AND ARTICLE OF MANUFACTURE 

FOR IP RADIO STREAM INTERCEPTION FOR 
NOTIFICATION OF EVENTS USING SYNTHESIZED AUDIO 

5 

BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention relates to the field of streaming content information over a data 

network such as the Internet. More specifically, the invention relates especially, but not exclusively, 

10 to handling of event-based messaging in conjunction with Internet streaming media. More 
specifically still, the invention relates especially, but not exclusively, to mixing event triggered 

yg messaging into preexisting Internet multi-media data streams such as Internet Radio data streams for 

yt delivery to a receiving device. 

H Description of the Related Art 

T5 [0002] It is known in the art to mix a first presentation of information with a second 

yh presentation of information from an independent source. For example, both United States Patent No. 
M 5,912,700, issued to Honey et al, and United States Patent No. 6,154,250, issued to Honey et al, 
^ teach a system for enhancing presentation of an object at a sporting event. Both Honey '700 and 

Honey '250 teach using one or more sensors to determine the position of the object and, based on 
20 the position and the field of view of a broadcast camera, editing or augmenting a television signal 

to enhance presentation of the object. Neither of these teach or suggest basing the editing or 

augmenting on event driven alerts, local buffering of messages or streams, or use of the Internet. 

[0003] Over the last several years, provision of audio and/or video streams over the Internet 

to receiving devices such as personal computers, WebTV™ receivers, or Internet phones has 
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increased. Recently, Internet-based radios ("IP Radios") and other so-called "Internet Appliances" 
have begun to be marketed as well. These audio and/or video streams can provide an experience 
equivalent to radio and/or television broadcasts. However, over the years radio and television 
broadcasts have been subject to a system of event-based interruptions such as during times of 
emergency. For example, United States Patent No. 5,495,283, issued to Cowe, teaches a cable 
television messaging system for selectively overlaying or substituting a desired message, for example 
an Emergency Broadcast System warning, onto multiple channels of a cable television system. 
Cowe c 283 does not teach or suggest basing the editing or augmenting on local event alerts, local 
buffering of messages or streams, or use of the Internet. No such system for event-based 
interruptions for Internet streams has been proposed to date. 

[0004] Incorporated by reference herein, United States serial no. 09/715,364 (attorney docket 

US 008064, filed 1 1/1 7/00 for Pieter van der Meulen ("van der Meulun '064")), for INTELLIGENT 
APPLIANCE HOME NETWORK, describes a set of consumer devices that can be used in 
automated home systems. These systems may incorporate alarm systems as well as status monitors 
and controllers for lights, motors, or appliances. Intelligent appliances may further provide 
mechanisms for bidirectional communications including the ability to receive commands and 
transmit status alerts. 

[0005] For systems that can respond to these status alerts by sending a user definable, user 

perceptible message, it is preferable for the message to affect ongoing audio and/or video streams, 
possibly without destroying the audio and/or video stream. It is also desirable to allow for a plurality 
of alerts such as those comprising status information, configurable by an end user. For example, 
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rather than limit the alerts to Emergency Broadcast System warnings, the end user may wish to have 
one or more intelligent appliances and/or other devices send alerts to one or more configurable 
receiving devices, including the same or different intelligent devices. 

[0006] Additionally, prior art systems do not allow configuring outbound messages that may 

5 arise as a result of alerts. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0007] Fig. 1 is a functional block diagrammatic representation of the present invention □ s 

apparatus; and 

D [0008] Fig. 2 is a flowchart of the present invention's monitoring software processes. 

J) DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

it" [0009] In general, throughout this description, if an item is described as implemented in 

£ software, it can equally well be implemented as hardware. 

O [0010] Referring now to Fig. 1, a schematic diagram of an exemplary local area network 190 

fH embodiment of the present invention, in one presently envisioned embodiment, a message triggered 
rf 5 by an event would interrupt a preexisting audio and/or video data stream, where the preexisting data 
stream may be discarded or stored for later playback such as with a Philips TIVO™ hard disk drive 
product. In another presently envisioned embodiment, a message triggered by an event would be 
superimposed over a preexisting audio and/or video data stream. One difficulty with such 
interruptions or superimposition occurs at the receiving side of the Internet audio and/or video data 
20 stream. Many systems today, especially wider bandwidth systems, are implemented using firewalls 
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or gateways (referred to generally as "monitor 195"), whether at home or in an office. These 
monitors 195 can isolate local area data to a local computer, e.g. 150, and/or local area network 190. 
[0011] As used herein the term "monitor" can have several configurations. By way of 

example and not limitation, monitor 195 may comprise a gateway as that term is understood by those 
skilled in the Internet arts. Monitor 195 may also comprise a standalone device such as clock-radio 
140 that is not connected to local area network 190 or that has its own connection to the Internet 
(identified generally as "104"). Further, connections to the Internet 104 may be via a dial-up modem 
with appropriate RJ-1 1 or equivalent connector, a digital subscriber link ("DSL") modem or router 
with appropriate RJ-11 or equivalent connector, or a cable modem or router having a coax cable 
connector for connecting to the Internet 104. Additionally, monitor 195 may have a local area 
network interface such as an ethernet adapter connector for connecting either to a single personal 
computer 150 or to local area network 190. Local area network 190 connections may comprise 
wired and/or wireless connections or a combination thereof. Monitors 195 may further implement 
a firewall such as with IP packet filtering, as these terms are familiar to those of ordinary skill in the 
Internet arts. Firewall configurations increase a user's safe connections to the Internet 104. 
[0012] By way of further example and not limitation, monitor 1 95 may additionally comprise 

network address translation ("NAT") capability. Monitors 195 that implement NAT allow multiple 
IP devices such as personal computers 150 or Internet appliances (e.g. coffee maker 110, washing 
machine 120, oven 130, or clock-radio 140) to connect simultaneously through monitor 195 out to 
the Internet 104 such as by way of local area network 190. In this configuration, monitor 195 is 
disposed intermediate the Internet 104 and local area network 190, including those devices connected 
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to local area network 190. In this manner, monitor 195 appears functionally as a single IP device, 
e.g. a single personal computer 150, to an Internet service provider. To accomplish this, monitor 195 
merges outgoing requests and sends incoming data to the appropriate IP device by keeping track 
internally of which packets are destined for which device (referred to herein as "targets" or 
5 "receiving devices"). 

[0013] In a further alternative configuration, monitor 195 may additionally possess bridging 

capabilities. Many data networks will likely be heterogeneous, and monitor 195 may be a focal point 
to bridge or connect these various networks. By way of example and not limitation, IEEE 1394 
p ("Firewire™") may be used to provide communication between two or more audiovisual 
If) components in a home, and those same components may connect as a source of audio Motion 
5 1 Pictures Expert Group ("MPEG") level 3 (MP3) for download or playback via monitor 195 to 
sJ personal computer 150 connected via local area network 190. 

O [0014] In a further alternative configuration, monitor 195 may additionally comprise a local 

ft| intranet or Internet server. Such a server may be always on and execute multiple, concurrent 
ft applications, by way of example and not limitation including world wide web services, other IP 
based services such as file transfer services, alarm systems, monitors for various home appliances, 
and/or camera capture systems such as those providing intruder detection capability. If so configured, 
these applications services may be reachable via the Internet 104 such as via a secure connection that 
prevents unauthorized access via tapping into the established IP stream by encrypting not only the 
20 data but also the login name and password provided by the remote user. 
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[0015] Personal computers 150 and Internet appliances such as clock-radio 140 may connect 

via the Internet 104 to a source of streaming audio such as Internet radio station 101, much like a 
traditional AM/FM radio "connects" to a broadcasting radio station. It is understood that local area 
network 190 could be located in an office, at a residence, or anywhere and is not constrained to 
connections to local area networks 190 located in a residence. As will be familiar to those of 
ordinary skill in the computer networking arts, local area networks 190 and data communications 
networks such as the Internet 104 comprise wired networks such as those utilizing topologies such 
as ethernet topologies, wireless networks such as Bluetooth networks, or any combination thereof 
[0016] As used herein, "Internet Radio" comprises IP packetized audio and/or video data 

streams such as streaming media as well as text and binary data streams. As will be familiar to those 
of ordinary skill in the Internet data communications arts, a flow of bytes is commonly referred to 
as a "stream," and the term "streaming media" is commonly used to describe content that is sent over 
the Internet 104 in a series of discrete data packets. As will be understood by those of ordinary skill 
in the Internet software arts, Internet Radio involves sending streaming media data from a server 
such as Internet Radio station 101 over the Internet 104 to a receiving device, typically a personal 
computer like personal computer 150 or an Internet appliance such as clock-radio 140. These 
streaming media data may be formatted in numerous formats including MP3, all of which will be 
familiar to those of ordinary skill in the data communications arts. 

[0017] Typically, streaming media content provided by Internet Radio station 101 will 

include voice and/or music. At times, streaming media data may be downloaded in advance to a 
cache local to the receiving device for faster or smoother playback, by way of example and not 
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limitation persistent data store 196 or a persistent data store 196 located at receiving stations like 
personal computer 150 or clock-radio 140. As used herein, persistent data store 196 is a device 
capable of maintaining programmatically accessible computer interrogatable data, by way of example 
and not limitation including magnetic hard disk drives, read-only memory devices including flash 
5 ROMs, solid state memories, tape devices, optical devices including CDROM and DVD-ROM, or 
any combination thereof. As further used herein, "hard disk drives" comprise both magnetic and 
optical media and their associated hardware. 

[0018] As further used herein, Internet Radio station 101 could be a traditional radio station 

r| which is additionally providing content over the Internet 104, an Internet-only station, or any 
fif combination thereof. The content is typically a continuous flow of bytes, which provides data at a 
CP constant rate (on average) and allows the content from Internet Radio station 101 to be conveyed to 
2 a listener. The content, streaming media, are typically transmitted to a web server (not shown in the 
□ figures) in a digitized and compressed format and then transmitted outward into the Internet 1 04 via 
nj communications link 103 to the Internet 104. Communication link 103 could be of any suitable 
IS type, such as high speed dial-up, integrated services digital network ("ISDN"), digital subscriber link 
("DSL"), Tl , T3, fiber-optic, wireless, and the like, or any combination thereof. 
[0019] As mentioned above, while the invention herein is described with respect to Internet 

Radio, it is equally applicable to other streamed media systems including those which can use local 
content from a jukebox, by way of example and not limitation video systems. An example of suitable 
20 local source for content in the video domain is a hard-disk based recorder, such as the Philips 
TIVO™ hard disk drive product. 
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[0020] As shown in Fig, 1, Internet appliances and other devices, discussed in van der Meulen ' 064, 
may be present behind monitor 195, by way of example and not limitation clock-radio 140, television 
160, coffee maker 110, washing machine 120, alarm system 170, and the like, or any combination 
thereof. These devices may be configured to provide an event-based alert 10 (not shown in the 
5 figures) intended for one or more monitoring devices such as monitor 195 or other receiving devices, 
including to one or more additional receiving devices connected locally to the sending device. As 
used herein, "event-based" comprises general status or alarm alerts. As also used herein, an "event" 
comprises connection requests, such as from personal computer 150 to Internet radio station 101; 
1*$ notifications, such as from clock-radio 140 indicating that it can deliver or receive audio; alerts, such 
if! as from alarm system 170 or appliance such as coffee maker 1 10; or any combination thereof. 
01 [0021] For example, as discussed in van der Meulen '064, a device such as washing machine 

J 1 20 may be configured to provide an alert 10 to another local device such as personal computer 1 50 
□ or television 160 that a laundry cycle has completed. An alarm system such as alarm 170 may be 
fy configured to provide an alert 10, by way of example and not limitation such as a door has been 
IS opened or a fire alarm triggered, to a local device such as personal computer 1 50 or television 1 60 
or to a remote device like Internet enabled cell phone 155. Coffee maker 1 10 may be configured to 
provide an alert 10 that coffee maker 1 10 is empty or is consuming abnormal current. 
[0022] In one embodiment current envisioned, these alerts 10 may be used by a home 

automation system to determine and/or communicate the state of each configured device as well to 
20 provide controls to these devices. In these situations, it is desirable to interrupt packet processing 
occurring at a receiving device such as personal computer 150, television 160, or clock-radio 140 
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to allow for notification of alert 10 and optionally for either automated or manual intervention. 
Additionally, alerts may be generated from devices outside local area network 190, e.g. a device 
connected to the Internet 104. By way of example and not limitation, it may be desirable to have a 
remotely monitored device such as alarm system 171 located at remote from a residence, e.g. at 
work, to notify a home-based system. 

[0023] On traditional radio or television stations, programs may be interrupted for messages 

based on external events, for example weather events that trigger the Emergency Broadcast System. 
Traditional radio or television programming may be interrupted by a variety of means, including 
tones, special displays, and like. The Internet radio stream may also be interrupted by the source of 
the data stream, e.g. Internet Radio station 101 may generate its own alerts. It may be desirable to 
disallow interruption of such Internet Radio station 101 generated interrupts, by way of example and 
not limitation when a remote radio station is alerting the presence of bad weather that is not 
applicable to a listener connected via the Internet 104. Mixing is the preferred embodiment, but 
interruption of the data stream is acceptable, especially where the original data stream will be 
buffered and played back later such as in an accelerated mode. 

[0024] In a currently preferred embodiment, monitor 1 95 is a device capable of independent 

data processing such as a general purpose personal computer or dedicated, specialized computer. 
Monitor 195 may be deployed intermediate the Internet 104 and devices connected to local area 
network 195, by way of example and not limitation devices such as television 160, personal 
computer 150, and clock-radio 140 connected via local area network 190 to each other and to 
monitor 195. Alternatively, monitor 105 may be part of television 160, personal computer 150, 
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and/or clock-radio 140 and not connected to local area network 190 such as with a standalone, 
Internet enabled clock-radio 140. 

[0025] In a presently preferred embodiment, monitoring software 197 resides and executes 

within monitor 195. However, monitoring software 197 may reside in totally within monitor 195, 
totally within any other device operatively connected to local area network 190 capable of executing 
monitoring software 197, or partially within any combination of monitor 195 and other devices 
operatively connected to local area network 190 capable of executing monitoring software 197 as 
will be familiar to those of ordinary skill in the distributed data processing arts. As used herein, 
"monitor 195" thus further comprises those devices in which monitoring software 197 is executing, 
as will be familiar to those of ordinary skill in the distributed data processing arts. 
[0026] Monitoring software 197 monitors events 30 (not shown in the figures) such as alerts 

10 from devices local to monitor 195, such as those connected to local area network 190. 
Monitoring software 197 further comprises access to a local persistent data storage device 196 that 
contains messages 20 (not shown in the figures) comprising a human perceptible message 20, by way 
of example and not limitation comprising messages 20 that are perceptible audibly, visually, 
tactilely, or any combination of these. Each message 20 may be associated with one or more alerts 
10 or events 30. Further, a provider or messages 20 and/or an authorized user may configure 
messages 20 as to content and other properties such as sound, color, destination receiving device, 
and the like, or any combination thereof. As will be understood by those of ordinary skill in the 
programming arts, local persistent data storage device 196 may further comprise a database such as 
a database of messages 20. 
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[0027] Although it is anticipated that many if not most messages 20 will be directed to 

receiving devices such as personal computer 150 or television 160 collocated with monitoring 
software 197, e.g. connected via local area network 190, monitoring software 197 may additionally 
have outgoing access to the Internet 104. A provider of messages 20 or an authorized user may thus 
configure monitoring software 197 and/or messages 20 to direct a predetermined set of messages 20 
to a specific destination external to local area network 190. By way of example and not limitation, 
one sending device may be alarm system 170. A user may configure monitoring software 197 to 
direct messages 20 responsive to alerts 10 from alarm system 170 to remote Internet enabled cell 
phone 155 via the Internet 104. 

[0028] In an alternative embodiment, monitoring software 197 may receive messages 20 

from an external source, such as from manufacturer 102 over the Internet 104. Manufacturer 102 
may therefore supply additional or modified messages 20 to be used by monitoring software 197 
with that manufacturer's devices. 

[0029] Referring now to Fig. 2, a flowchart showing the general process of the present 

invention, monitoring software 197 monitors all packets received at monitor 195 and determines, 
at 200, if the packets contain Internet data streams and/or events 30. In a preferred embodiment, if 
monitoring software 197 receives an event 30 comprising an alert 10, monitoring software 197 may 
interrupt the Internet radio stream and substitute message 20 into the Internet radio stream 254 for 
the intended receiving device. In an alternative embodiment, monitoring software 197 may mix 
message 20 into the ongoing Internet radio stream 254 intended for delivery to a receiving device, 
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such as by dynamically lowering the amplitude of the Internet radio stream audio while providing 
message 20 at an amplitude perceived as louder by a human listener at a receiving device. 
[0030] In a further alternative embodiment, monitoring software 197 may buffer the ongoing 

Internet radio stream intended for delivery to a receiving device, such as by storing it to persistent 
data store 196, send message 20 to a receiving device, and then resume delivery of the buffered 
Internet radio stream at the point where they were suspended. In these embodiments, data may be 
buffered for delayed playback. Buffered data can be played back in an accelerated mode, e.g. 25% 
faster. However, in general the buffering of the data stream needs to continue for a time period that 
is longer than the time that was needed for the original message. Buffering will generally continue 
until the accelerated stream has caught up with the original broadcast. 

[0031] It is to be noted that messages 20 can be sent to more than one receiving device and 

may be sent to all available receiving devices in case of an high priority message like an alarm; new 
alerts 10 may be received via the Internet 104 from a "trusted" site, such as the vendor of an Internet 
appliance, e.g. an Internet enabled washing machine 120; and voices can be prerecorded, by way of 
example and not limitation by the vendor, spoken in by the user, or synthesized from the text 
representing the message. 

[0032] In the currently preferred embodiment, messages 20 are available in a plurality of 

formats, by way of example and not limitation including text, audio, video, vibration pattern like 
with pagers, and the like, or any combination thereof. It is also noted that some receiving devices 
may be capable of rendering message 20 in different ways. By way of example and not limitation, 
television 160 can do audio and video, but clock-radio 140 would ignore all but the audio. However, 
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if clock-radio 140 also has a display such as an LCD text display, clock-radio 140 could also show 
the text. 

[0033] In the operation of the preferred embodiment, referring to Fig. 1 and Fig. 2, 

monitoring software 197 continually monitors packets received at monitoring software 197. As used 
5 herein, packets, as will be understood by those of ordinary skill in the programming arts, may contain 

specific data that are events 30 or alerts 10 as well as Internet Radio data streams. 

[0034] Certain packets require no additional processing such as packets already destined for 

a receiving device. Other packets may be intended for a receiving device for which routing has not 
^ yet been accomplished, and monitoring software 197 may establish that routing 216, e.g. clock-radio 
If) 1 40 requesting a new connection to Internet radio station 101. 

J [0035] In the absence of an event 30, monitoring software 197 determines if the received 

J packet requires manipulation 250, by way of example and not limitation if monitoring software 197 
q is already processing alert messages for the desired receiving device. If not, monitoring software 197 
f!| may then allow a received packet to be transmitted without further processing 252 to a target such 
$5 as clock-radio 140 or personal computer 150. 

[0036] When an event 30 is detected, at 210, by monitoring software 197, monitoring 

software 1 97 determines if the event 30 requires notification 220 to be sent to one or more receiving 
devices. In certain situations, event 30 notification is of an informational type. By way of example 
and not limitation, such an event 30 may be from a device such as clock-radio 140 notifying 
20 monitoring software 197 or another device of the ability of clock-radio 140 to receive or deliver 
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audio streams. For these events 30, monitoring software 197 may add the device 222 as a target for 
packets with a set of properties appropriate for that device, e.g. can process audio streams. 
[0037] Certain events 30 require additional processing. Monitoring software 197 attempts 

to match 230 these events 30 with a predetermined response such as with an audio or audio-visual 
message 20 in a database resident in persistent data store 196. Monitoring software 197 retrieves 
the message 20 such as at step 230 for further processing. 

[0038] In one currently envisioned embodiment, messages 20 may have properties 

associated with the messages 20 to instruct monitoring software 197 on handling of those messages 
20, by way of example and not limitation including how to handle concurrent streams such as 
lowering or muting volume or dimming background displays, sounding predetermined tones, 
repetition counters, further actions required, destination receiving devices, or the like, or any 
combination thereof. 

[0039] In a preferred embodiment, if monitoring software 197 determines that the receiving 

device is also a current destination for one or more Internet radio data streams 250, monitoring 
software 197 may manipulate the Internet radio data stream according to preconfigured or dynamic 
rules. By way of example and not limitation, monitoring software 197 may mix the associated 
message 20 into the Internet radio data stream after lowering the volume attribute of the Internet 
radio data stream. By way of further example and not limitation, monitoring software 197 may 
temporarily suspend the Internet radio data stream, buffering the Internet radio data stream for 
resumption such as upon a predetermined triggering event such as a user acknowledgment. 


14 


US018053 

[0040] Monitoring software 197 retrieves all receiving device addresses associated with 

event 30 for notification at step 235. If additional receiving devices are found 240, message 20 is 
sent to each additional receiving device. If additional received packets are slotted for transmission 
to those receiving devices, monitoring software 197 processes message 20 with the received packets 
250 before sending 254 the processed packets to the receiving devices. 

[0041] If no receiving device is found for message 20 at step 240, monitoring software 197 

may be configured to support a default receiving device and send message 20 to the default receiving 
device 245. In the currently preferred embodiment, there is a configurable default receiving device 
for each message 20. By way of example and not limitation, a speaker that currently is a component 
of most personal computers 150 could act as the default receiving device for messages 20 destined 
for monitor 195. Similarly, if there is no known message associated with an event, message 20 may 
default to a predetermined message 20 such as a message 20 comprising text or synthesized audio 
or both. By way of example an not limitation, default message 20 might comprise information such 
as "An alarm has been received from [OOdevice]" where "[OOdevice]" is the device that generated 
the event. 

[0042] It will be understood that various changes in the details, materials, and arrangements 

of the parts which have been described and illustrated above in order to explain the nature of this 
invention may be made by those skilled in the art without departing from the principle and scope of 
the invention as recited in the following claims. 
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